Repository navigation
docs: split the site into tutorial, how-to, explanation and reference - #21
Merged
Merged
Conversation
The site was a Sphinx build of Markdown pages under docs/, with the Python API from autodoc, which imported the guard to read it, and nothing from the Zig build. Nothing checked its links, and a page could name a function that no longer existed. The documentation template builds it now. Org pages under docs/orgmode/ are exported to reStructuredText and built by Sphinx with warnings and unresolved names as errors. docs/tools/zig_api.zig writes the API pages from the doc comments of zig/main.zig and the files it imports, sphinx-autoapi reads src/eessi_arch_guard.py without importing it, and zig build docs installs the Zig autodoc site, from a compile of its own so that the binary is not built with -femit-docs. check_site.py checks the internal links and anchors, and that no page names a site; lychee checks the external links. pixi.lock pins the tools, so dependabot no longer watches docs/requirements.txt. The first pages are the tutorial, "See the guard reload EESSI for another CPU", run with Python 3.12.3 as the page says; the how-to on installing the prolog; the explanation of how the guard decides, with two drawings rendered from the Graphviz sources beside them; and the compatibility table. The contributing page says how to record a change and run the tests, and links RELEASING.md. CHANGELOG.md gets a title and the marker under which towncrier adds a release. The docs workflow replaces pages.yml: it builds the site as docbld does, checks the external links, also weekly, and publishes from main. The CI docs job keeps running the example scripts the pages quote.
The API pages list each declaration with its doc comment, and 38 of the 65 pub declarations of the Zig build had none, nor did 25 of the 66 public functions, classes and methods of the Python guard. Each has one now, written from the code: the outcome kinds, the deadline and what stops it, the probes and what each status means, the cache and the restore helper. A field of a public struct says what it holds where its name does not. The comment on eessiModule gave its pattern as plain text, which the Zig API page passed through with <segment> as an HTML tag; it is code now. Comments and docstrings only: the release binaries built with -Dcommit=a418152 have the same sha256 as before for x86_64-linux-musl and aarch64-linux-musl, and the syntax tree of src/eessi_arch_guard.py without its docstrings is unchanged.
Run side by side on the same inputs, the Python guard and the Zig build differ in four cases that no row of COMPAT.md allows, and share one bug: - Started through PATH by a symlink, the Zig build reads the config beside the symlink's target, as /proc/self/exe names it; the Python guard reads it beside the symlink. - When the deadline passes and the cache directory cannot be opened for the late cache read, the Python guard prints a skip line with the PermissionError; the Zig build reports the timeout. - A --restore request whose env holds a list, an object, NaN or Infinity is refused by the Zig build; the Python guard takes str() of the value. - A timeout above 2147483.647 seconds passes the check, but the Python guard then fails every decision that runs a command with an OverflowError from poll(); the Zig build decides. - In both, --chain takes --strict as its path, so --chain --strict is not strict, though the strict workspace rule applies. Two new tables list them. The cache row now gives the digest the code computes, with the task's processors and visible GPUs, and no longer says the mode is outside the cache key while the name holds it. The rows on the config path, the timeout and the deadline point at the new tables. The code is unchanged.
DESIGN.md still described the guard before the fixes since it was written. Read against the code, it now says that: - archdetect runs on the processors the task may run on; - the helper is python3 -I -X utf8 FILE --restore, with --strict when the guard has it, and its environment also holds EESSI_ARCH_GUARD_WORKDIR, one new private directory with HOME and TMPDIR; - no prolog line may pass the 4094 bytes Slurm 17.11 reads; - a four-digit year such as 2026 is a module name; - --strict, for a user's job wrapper, exits 78 for a failed decision, an unexpected exception, a timeout or SIGTERM, while a site prolog, started with no arguments, keeps exit 0; - a timeout above 2147483.647 seconds fails the Python guard's decisions, with a pointer to COMPAT.md; - the cache is per output mode, keys on the task's processors and visible GPUs, lives in the private workspace under --strict, and trusts only a regular file owned by the caller with one link and no group or other bits; - only local task 0 prints the prolog line; - the Zig build is a static binary beside the Python file, which stays the default. COMPAT.md's row on --chain --strict said an unexpected exception exits 78. In the decision it exits 0, as a failed decision does; only an error the decision's handler does not catch, such as a crash of the Zig build, exits 78. The row now says so. The code is unchanged.
The site had the tutorial, one how-to, one explanation, Compatibility and the API. The rest of the old Markdown pages are now Org pages in the same four parts, read against the code: - how-to: configure the guard, use it without changing the site TaskProlog, run it in a batch script, run a program you built yourself on more than one node type, and correct a failed reload; - explanation: why a task prolog runs the reload, and prolog lines and shell lines, with the decision shared per step and the deadline; - reference: the command, the configuration file, the environment variables, the messages and markers, and the CPU flags. The transcripts are from docs/examples/ and from runs of the guard installed as /usr/local/sbin/eessi-arch-guard with a root-owned configuration file. Where the old pages had drifted from the code they now say what it does: the helper's environment, the accelerator output rules, every outcome but a timeout, SIGTERM or exception being stored, and a stored decision that changes an LD_ variable not being reused. "How the guard decides" has the headings of the other explanation pages, italic leads without code in them, and links to the API of both builds. The tutorial and the install guide link the new pages, and the toctrees list them.
COMPAT.md said the second configuration file is looked for beside the path the program was started by. The code passes __file__, which is that path for the file itself but the module in site-packages for the console script pip installs; the row now says so. DESIGN.md listed the helper's environment as PATH, HOME, LANG, LC_ALL, TMPDIR and EESSI_ARCH_GUARD_WORKDIR and said the last three name its private directory. HOME, TMPDIR and EESSI_ARCH_GUARD_WORKDIR do; the list is now in that order.
pub fn build in build.zig and pub fn main in docs/tools/zig_api.zig were the two pub declarations left without a doc comment. Both now say what they do. The release binaries of both targets are byte-identical to those built before.
The environment variables page said that EESSI_PREFIX and EESSI_INIT_PREFIX must be absolute paths, and that TMPDIR holds the stored decision when it is an absolute directory. Both builds check more than a leading slash: safe_absolute in the Python guard and safeAbsolute in the Zig build also refuse a path with a .. part, a newline, a carriage return or a NUL. With a .. part TMPDIR is not used, and an EESSI_PREFIX ends in archdetect-failed in both builds, although the directory exists; EESSI_INIT_PREFIX is checked the same way. The page now defines the term once, under the table of the variables read from the task. The --chain path is outside it: that check allows a .. part, and the command page already says what it refuses.
"Run the guard in a batch script" quotes the output of --strict --shell with no private workspace, but no script in the repository printed it: it came from a script run once outside the tree, so a reader could not run it again. docs/examples/strict_workspace.py prints it now, on the stack the other examples build, and then checks that the reload runs once ~/.local/state/eessi-arch-guard exists with mode 700, as step 4 of the guide creates it. Both builds print the same transcript. The guide names the script and shows its command, as its check section does for shell_demo.py, and CI's docs job runs it with the other examples. Every output block the pages quote now comes from a script in the repository.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The documentation site is in four parts, as Diátaxis has them: the tutorial "See the guard reload EESSI for another CPU", how-to guides, explanation and reference, 34 pages in all. Org pages under
docs/orgmode/are exported and built by Sphinx. The API pages come from the doc comments ofzig/main.zigand the files it imports, and from the docstrings ofsrc/eessi_arch_guard.py. Everypubdeclaration, and every public function, class and method of the Python guard, has one.The transcripts the pages quote come from the scripts in
docs/examples/, which CI's docs job runs.docs/examples/strict_workspace.pyprints the--strict --shellfailure that "Run the guard in a batch script" quotes, then checks the reload once~/.local/state/eessi-arch-guardexists.The release binaries of both targets have the same sha256 at this tip as at the base. Behaviour is unchanged.
COMPAT.md lists five known code bugs, left for follow-up: a
--timeoutabove 2147483.647 s overflows in the Python guard; the Zig build reads its config beside a symlink's target when started throughPATH; a deadline against an unopenable cache directory is reported as a skip in the Python guard and as a timeout in the Zig build; a--restorerequest whoseenvholds a value that is not a string is handled differently in the two builds; and--chain --stricttakes--strictas the path, so the run is not strict.